Display control device, display control method, and storage medium storing display control program

ABSTRACT

A display control device includes processing circuitry to acquire traveling environment information; to estimate driving skill of the driver; to detect a direction of a line of sight of the driver; to accumulate information on movements of the sight line direction and driving action information; to calculate a visual field range and a visual recognition level indicating ease of recognition of each of a plurality of sections in the visual field range by the driver&#39;s vision based on the information on the movements of the sight line direction and the driving action information accumulated in the memory and at least one of the traveling environment information and the driving skill information; and to receive display information and to control at least one of a display position and a display method of an image on a basis of the display information.

TECHNICAL FIELD

The present invention relates to a display control device, a displaycontrol method and a display control program.

BACKGROUND ART

There has been proposed a device that measures a visual field range as arange visible to a driver of a vehicle based on the direction of thedriver's line of sight after movement when the movement of the line ofsight of the driver situated on the driver's seat of the vehicle isdetected and projects an image onto the inside of the visual field rangeon the front windshield (windshield) (see Patent Reference 1, forexample).

PRIOR ART REFERENCE Patent Reference

Patent Reference 1: Japanese Patent Application Publication No.2010-18201

SUMMARY OF THE INVENTION Problem to be Solved by the Invention

However, the visual field range of the driver changes depending onfactors other than the direction of the driver's line of sight. Forexample, the visual field range of the driver has a tendency to narrowwhen the driver's driving skill (driving proficiency) is low orenvironment around the traveling vehicle (e.g., congestion level) is athigh density. Therefore, if the visual field range is determined basedexclusively on the direction of the line of sight, there is apossibility that the driver overlooks the image displayed in the visualfield range.

An object of the present invention, which has been made to resolve theabove-described problem with the conventional technology, is to providea display control device, a display control method and a display controlprogram capable of controlling a display device so that an image isdisplayed at a position appropriate for the driver.

Means for Solving the Problem

A display control device according to an aspect of the present inventionincludes an environment information acquisition unit that acquirestraveling environment information indicating environment around avehicle driven by a driver; a skill estimation unit that estimatesdriving skill of the driver and generates driving skill informationindicating the driving skill, a sight line detection unit that detects adirection of a line of sight of the driver and generates sight linedirection information indicating the direction of the line of sight; avisual recognition level calculation unit that calculates a visual fieldrange indicating a region that can be visually recognized by the driverand a visual recognition level indicating ease of recognition of each ofa plurality of sections in the visual field range by the driver's visionbased on the sight line direction information and at least one of thetraveling environment information and the driving skill information; anda display control unit that receives display information to be presentedto the driver and controls at least one of a display position and adisplay method of an image on a basis of the display information basedon the display information, the visual field range and the visualrecognition level.

A display control method according to another aspect of the presentinvention includes a step of acquiring traveling environment informationindicating environment around a vehicle driven by a driver; a step ofestimating driving skill of the driver and generating driving skillinformation indicating the driving skill; a step of detecting adirection of a line of sight of the driver and generating sight linedirection information indicating the direction of the line of sight; astep of calculating a visual field range indicating a region that can bevisually recognized by the driver and a visual recognition levelindicating ease of recognition of each of a plurality of sections in thevisual field range by the driver's vision based on the sight linedirection information and at least one of the traveling environmentinformation and the driving skill information; and a step of receivingdisplay information to be presented to the driver and controlling atleast one of a display position and a display method of an image on abasis of the display information based on the display information, thevisual field range and the visual recognition level.

Effect of the Invention

According to the present invention, the display control can be executedso that the image is displayed at a position appropriate for the driver.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing an example of the hardware configuration ofa driving support device including a display control device according toa first embodiment of the present invention.

FIG. 2 is a diagram showing an example of images displayed based oncontrol executed by the display control device according to the firstembodiment.

FIG. 3 is a functional block diagram schematically showing theconfiguration of the display control device according to the firstembodiment.

FIG. 4 is a flowchart showing the operation of the display controldevice according to the first embodiment.

FIG. 5 is a diagram showing an example of a visual recognition level mapindicating visual recognition levels used by the display control deviceaccording to the first embodiment.

FIG. 6 is a diagram showing an example of an image displayed based oncontrol executed by the display control device according to the firstembodiment.

FIG. 7 is a diagram showing an example of an image displayed based oncontrol executed by the display control device according to the firstembodiment.

FIG. 8 is a diagram showing an example of an image displayed based oncontrol executed by a display control device according to a secondembodiment of the present invention.

FIG. 9 is a diagram showing an example of an image displayed based oncontrol executed by the display control device according to the secondembodiment.

FIG. 10 is a diagram showing an example of an image displayed based oncontrol executed by a display control device according to a thirdembodiment of the present invention.

MODE FOR CARRYING OUT THE INVENTION

A display control device, a display control method and a display controlprogram according to each embodiment of the present invention will bedescribed below with reference to the drawings. The followingembodiments are just examples and a variety of modifications arepossible within the scope of the present invention.

(1) First Embodiment (1-1) Driving Support Device 1

FIG. 1 is a diagram showing an example of the hardware configuration ofa driving support device 1 including a display control device 100according to a first embodiment of the present invention. As shown inFIG. 1, the driving support device 1 is installed in an automobile as avehicle driven by a driver. The driving support device 1 includes ahead-up display (HUD) device 11, an instrument panel display device 12,electronic side mirror display devices 13 and 14, an electronic rearviewmirror display device 15, an in-vehicle camera 21, a vehicle-outsidecamera 22 and the display control device 100. The driving support device1 may include a microphone (not shown) for detecting conversationalvoice in the vehicle. The display control device 100 includes aprocessor 31 and a memory 32. The HUD device 11 displays various itemsof information in superimposition with a scene viewed through a frontwindshield 10, for example. The HUD device 11 displays a semitransparentimage, for example. The instrument panel display device 12 is aninstrument panel including a speedometer and the like, which is referredto also as a panel display device 12. The electronic side mirror displaydevices 13 and 14 are display devices for displaying images of scenes tothe left and right rear of the vehicle captured by the vehicle-outsidecamera 22, for example, which are referred to also as side mirrordisplay devices 13 and 14. The electronic rearview mirror display device15 is a display device for displaying an image of a scene to the rear ofthe vehicle captured by the vehicle-outside camera 22, for example,which is referred to also as a rearview mirror display device 15. Thedriving support device 1 may include a sensor such as an infrared raysensor for detecting or measuring environment outside the vehicleinstead of or in addition to the vehicle-outside camera 22.

The HUD device 11 is a display device that displays an image insuperimposition with the scene outside the vehicle viewed through thefront windshield of the vehicle. The panel display device 12 is adisplay device in the instrument panel arranged under the frontwindshield. The side mirror display device 13 is a display device thatdisplays an image of a scene outside the vehicle at a position on aleft-hand side of the panel display device 12. The side mirror displaydevice 14 is a display device that displays an image of a scene outsidethe vehicle at a position on a right-hand side of the panel displaydevice 12. The rearview mirror display device 15 is a display devicethat displays the environment to the rear of the vehicle. While fivedisplay devices are shown in FIG. 1, the number of display devices isnot limited to five.

The in-vehicle camera 21 is an image capturing device that capturesimages of the driver seated on the driver's seat of the vehicle. Thevehicle-outside camera 22 is an image capturing device that capturesimages of the scene outside the vehicle, that is, around the vehicle.While one in-vehicle camera 21 is shown in FIG. 1, there can be two ormore in-vehicle cameras. It is unnecessary to provide the in-vehiclecamera 21 when another means for detecting the driver's line of sight isprovided. Further, while one vehicle-outside camera 22 is shown in FIG.1, there can be two or more vehicle-outside cameras.

The display control device 100 controls the operation of the HUD device11, the panel display device 12, the side mirror display devices 13 and14, the rearview mirror display device 15, the in-vehicle camera 21 andthe vehicle-outside camera 22. Further, the display control device 100acquires information regarding movement of the vehicle, informationregarding driving operations by the driver, and so forth from a movementcontrol device (not shown) that controls the movement of the vehicle.The information regarding the movement of the vehicle includes travelingspeed, travel distance, fuel remaining, and so forth. The informationregarding the driving operations includes condition of a brake operationby the driver, condition of a steering operation by the driver, and soforth.

The display control device 100 includes the processor 31 as aninformation processing unit and the memory 32 as a storage unit forstoring information. The display control device 100 is a computer, forexample. A display control program for executing a display controlmethod has been installed in the memory 32. The display control programis installed via a network or from a storage medium storing information,for example. The processor 31 controls the operation of the whole of thedriving support device 1 by executing the display control program storedin the memory 32. The whole or part of the display control device 100may be formed with a control circuit made up of semiconductor integratedcircuits. The memory 32 may include various types of storage devicessuch as a semiconductor storage device, a hard disk drive and a devicethat records information in a removable record medium.

FIG. 2 is a diagram showing an example of images displayed based oncontrol executed by the display control device 100 according to thefirst embodiment. FIG. 2 indicates the positions of the images displayedin front directions as viewed from the driver situated on the driver'sseat of the vehicle. The HUD device 11 displays an image insuperimposition with the scene outside the vehicle viewed through thefront windshield 10 of the vehicle. The panel display device 12 isarranged under the front windshield 10. The side mirror display devices13 and 14 are respectively arranged to the left and right of the paneldisplay device 12. The rearview mirror display device 15 is arranged onan upper part of the front windshield 10. However, the arrangement andshapes of these display devices are not limited to the example of FIG.2.

(1-2) Display Control Device 100

FIG. 3 is a functional block diagram schematically showing theconfiguration of the display control device 100 according to the firstembodiment. The display control device 100 controls display content,display positions and display methods of the images presented to thedriver by the various types of display devices shown in FIG. 2. As shownin FIG. 3, the display control device 100 includes a skill estimationunit 101, a sight line detection unit 102, an environment informationacquisition unit 103, a visual recognition level calculation unit 104and a display control unit 105.

The environment information acquisition unit 103 acquires travelingenvironment information D3 indicating the environment around the vehicledriven by the driver. The skill estimation unit 101 estimates drivingskill of the driver driving the vehicle and generates driving skillinformation D1 indicating the driving skill. The sight line detectionunit 102 detects the direction of the line of sight of the driverdriving the vehicle and generates sight line direction information D2indicating the direction of the line of sight.

The visual recognition level calculation unit 104 calculates a visualfield range indicating a region that can be visually recognized by thedriver at the present time point and a visual recognition level D4 ineach of a plurality of sections (i.e., a plurality of subregions) in thevisual field range based on the sight line direction information D2 andat least one of the traveling environment information D3 and the drivingskill information D1. The visual recognition level D4 is a valueindicating to what extent the driver easily visually recognizes each ofthe plurality of sections in the visual field range, that is, ease ofvisual recognition of each of the plurality of sections. The visualfield range, the plurality of sections and the visual recognition levelD4 will be described later by using FIG. 5. Incidentally, the followingdescription will be given of an example in which the visual recognitionlevel calculation unit 104 calculates the visual field range and thevisual recognition level D4 in each of the plurality of sections in thevisual field range based on the sight line direction information D2, thetraveling environment information D3 and the driving skill informationD1.

The display control unit 105 receives display information D0 to bepresented to the driver and controls at least one of the displayposition and the display method of an image on the basis of the displayinformation based on the display information D0 and the information D4indicating the visual recognition levels in the visual field range. Thedisplay control unit 105 executes display control for presenting eachdriver with the image including the display information D0 at a positionsuitable for each driver. The display information D0 includesinformation provided from the movement control device (not shown) thatcontrols the movement of the vehicle, information provided from anavigation device installed in the vehicle, information received by acommunication device installed in the vehicle, and so forth.

FIG. 4 is a flowchart showing the operation of the display controldevice 100 according to the first embodiment. First, in step S1, thedisplay control device 100 acquires the traveling environmentinformation D3 indicating the environment around the vehicle driven bythe driver. In step S2, the display control device 100 estimates thedriving skill of the driver driving the vehicle and generates thedriving skill information D1 indicating the driving skill. In step S3,the display control device 100 detects the direction of the line ofsight of the driver driving the vehicle and generates the sight linedirection information D2 indicating the direction of the line of sight.The order of the processes of the steps S1 to S3 is not limited to thisorder. The processes of the steps S1 to S3 may also be executed inparallel with each other.

In step S4, the display control device 100 calculates the visual fieldrange indicating the region that can be visually recognized by thedriver at the present time point and the visual recognition level D4 ineach of the plurality of sections in the visual field range based on thesight line direction information D2 and at least one of the travelingenvironment information D3 and the driving skill information D1.

In step S5, the display control device 100 receives the displayinformation D0 to be presented to the driver and controls at least oneof the display position and the display method of the displayinformation D0 based on the display information D0 and the visualrecognition level D4 in each of the plurality of sections in the visualfield range. The display control device 100 executes the display controlfor presenting the image including the display information D0 at theposition suitable for each driver.

(1-3) Skill Estimation Unit 101

The skill estimation unit 101 estimates the driving skill of the driverbased on driving actions of the driver and generates the driving skillinformation D1 indicating the driving skill. In other words, the skillestimation unit 101 estimates the driving skill, namely, drivingproficiency, of the driver based on driving operations performed by thedriver and generates the driving skill information D1 indicating thedriving skill. The driving skill is referred to also as a drivingempirical value since the driving skill varies depending on the amountof driving experience. The driving operation used for the estimation ofthe driving skill is, for example, a hard braking operation, a steeringoperation, a gear shifting operation, or a combination of two or more ofthese operations. However, the driving operation used for the estimationof the driving skill is not limited to the above-described operations orcombinations.

The skill estimation unit 101 judges that the hard braking operation hasbeen performed by the driver when the degree of depressing the brakepedal is higher than or equal to a depression threshold value determineddepending on the driving speed. The skill estimation unit 101 canestimate that the driving skill of the driver is low when the ratio ofthe number of times of the hard braking operation to the number of timesof the braking operation in the driving is higher than a predeterminedreference ratio and estimate that the driving skill of the driver ishigh when the ratio of the number of times of the hard braking operationis less than or equal to the reference ratio. However, the driving skillestimation method based on the braking operation is not limited to thismethod. While the driving skill is classified into two groups based onthe braking operation in the above-described example, it is alsopossible to classify the driving skill into three or more groups basedon the braking operation.

The skill estimation unit 101 can also estimate that the driving skillof the driver is high when a revolution speed of the steering wheel inthe steering operation is higher than a predetermined reference speedand estimate that the driving skill of the driver is low when therevolution speed of the steering wheel is less than or equal to thepredetermined reference speed. However, the driving skill estimationmethod based on the steering operation is not limited to this method.While the driving skill is classified into two groups based on thesteering operation in the above-described example, it is also possibleto classify the driving skill into three or more groups based on thesteering operation.

The skill estimation unit 101 can also accumulate history records of thegear shifting operation in regard to each of predetermined speed rangesof the vehicle's traveling speed and estimate the driving skill of thedriver by referring to the accumulated operation history records. Forexample, the skill estimation unit 101 can estimate that the drivingskill of the driver is high when the ratio of the number of times ofperforming gear shifting suitable for each speed range to the totalnumber of times of gear shifting is higher than a predeterminedreference ratio and estimate that the driving skill of the driver is lowwhen the ratio is less than or equal to the reference ratio.

The skill estimation unit 101 can also previously store past driverinformation, such as the sum total of driving travel distances of thedriver, information on past traffic accidents that occurred to vehiclesdriven by the driver and records of past traffic violations of thedriver, in a storage unit and estimate the driving skill by referring tothese items of information. The storage unit is the memory 32 shown inFIG. 1, for example. The previous inputting of the driver information tothe storage unit of the skill estimation unit 101 can be carried out asinputting by manual operations by the user, inputting by a card readingdevice by reading in data recorded in an IC card or the like, performingindividual identification of the driver based on information outputtedfrom the in-vehicle camera 21 that captures images of the inside of thevehicle and acquiring data existing on a network or in-vehicle databased on the individual identification information on the driver, and soforth. The storage unit can also be a storage device outside the displaycontrol device 100 or an external storage device communicativelyconnected to the display control device 100 via a network.

Further, the driving skill information D1 changes depending on thecondition of the driver. For example, the skill estimation unit 101 canestimate present health condition of the driver from biologicalinformation such as the heart rate, complexion or sight line movement ofthe driver and correct the estimated driving skill to the driver'soriginal driving skill based on the present health condition. The skillestimation unit 101 can also detect the driver's actions other than thedriving actions, such as conversation between the driver and a passengerand inattentive driving by the driver looking aside, and correct thedriving skill based on the driver's actions other than the drivingactions.

The skill estimation unit 101 can also make a correction under aparticular condition by storing previously collected information on thedriver's health condition such as chronic diseases or eyesight of thedriver as the driving skill information D1 by reading in from an ICcard, manual inputting or the like.

The skill estimation unit 101 may estimate the driving skill by using acombination of two or more of the various driving skill estimationmethods described above. For example, the skill estimation unit 101 maydetermine the driving skill information based on one or more items ofinformation among the driver's driving experience, driving historyrecords, health condition and eyesight information.

(1-4) Sight Line Detection Unit 102

The sight line detection unit 102 obtains the sight line directioninformation D2 indicating the direction of the line of sight of thedriver. The sight line detection unit 102 obtains the sight linedirection information D2 indicating the direction in which the driver isviewing. The direction in which the driver is viewing is the directionin which the driver is looking with the eyes. The direction in which thedriver is viewing is referred to also as a “sight line direction”. Thesight line detection unit 102 detects the sight line direction of thedriver based on an image of the driver's face or eyes captured by thein-vehicle camera 21. The sight line detection unit 102 is capable ofdetecting the driver's sight line direction three-dimensionally by useof three-dimensional measurement based on the positions of pupils ofboth eyes of the driver with respect to the position of the face of thedriver in a captured image captured by the in-vehicle camera 21. Thesight line detection unit 102 can also be a part of a device worn by thedriver. The sight line detection unit 102 can be a device provided as apart of eyeglasses worn by the driver, for example.

(1-5) Environment Information Acquisition Unit 103

The environment information acquisition unit 103 acquires the travelingenvironment information D3 indicating the environment around the vehiclebased on images captured by the vehicle-outside camera 22. The travelingenvironment information D3 includes information indicating objectsexisting around the vehicle driven by the driver and the situationaround the vehicle. The environment information acquisition unit 103acquires information regarding the objects existing around the vehiclefrom the vehicle-outside camera 22 provided on the vehicle. The objectsexisting around the vehicle are, for example, other vehicles,pedestrians, animals, obstacles, and so forth. The situation around thevehicle includes, for example, information indicating weather acquiredbased on vehicle position information and time information, informationindicating the direction of sunlight, road congestion informationacquired based on the vehicle position information and the timeinformation, and so forth. The environment information acquisition unit103 may acquire the weather and road environment (e.g., congestioncondition) from a server for information provision via a network.Further, the environment information acquisition unit 103 may acquirethe position information on the vehicle driven by the driver from apositioning system such as a GPS (Global Positioning System).

(1-6) Visual Recognition Level Calculation Unit 104

FIG. 5 is a diagram showing an example of a visual recognition level map40 indicating the visual recognition levels D4 generated by the visualrecognition level calculation unit 104 of the display control device 100according to the first embodiment. The visual recognition level D4 isinformation quantifying a character representing the ease of visualrecognition of each of the sections in the visual field range of thedriver. The visual recognition level D4 in each of the plurality ofsections in the visual field range can be represented by the visualrecognition level map 40. The visual recognition level map 40 is formedof a plurality of rectangular sections arranged in a grid and the visualrecognition level in each of the plurality of rectangular sections. Thevisual recognition level map 40 shown in FIG. 5 indicates the amount ofinformation that can be visually recognized by the driver when thedriver views forward in the state of the present sight line direction,by using densities and different fill-in patterns of the plurality ofsections. For example, with the increase in the color density of asection in the visual recognition level map 40, the driver can easilyacquire visual information, whereas guidance of the line of sight ismore likely to occur. Namely, a section in dense color is a region withhigh perceptual sensitivity and has high ability to guide the line ofsight towards the section. With the decrease in the color density of asection in the visual recognition level map 40, the section is lesslikely to hinder the driver's driving actions, whereas the driver ismore likely to overlook the displayed image. Namely, a section in palecolor is a region with low perceptual sensitivity and has low ability toguide the driver's line of sight towards the section.

The visual recognition level calculation unit 104 calculates the visualrecognition level D4 based on the sight line direction information D2,the driving skill information D1 and the traveling environmentinformation D3. For the calculation of the visual recognition level D4,the visual recognition level calculation unit 104 can employ a method ofestimating regions of central vision and peripheral vision used in thefield of human visual cognitive engineering. The central vision regionis a region close to the line of sight in the visual field range and isa region that is easily perceived by the driver. In other words, thecentral vision region is a visual region with high perceptualsensitivity. The peripheral vision region is a region far from the lineof sight in the visual field range and is a region that is difficult tobe perceived by the driver. In other words, the peripheral vision regionis a visual region with low perceptual sensitivity even though beingvisible to the driver. Incidentally, magnitude of the perceptualsensitivity at a boundary between the central vision region and theperipheral vision region and in each of the regions in the visual fieldrange varies depending on physical individual differences, environmentaround the driver, and so forth.

The visual recognition level calculation unit 104 estimates the visualfield range based on the driving skill information D1 and the sight linedirection information D2. When the driver has a lot of drivingexperience, that is, when the driver has high driving proficiency, thedriver has a wide visual field and thus the visual field range is wide.The reason why the visual field range is wide is that a driver havinghigh proficiency empirically has thorough knowledge of visualinformation necessary for performing a driving action and thus neverfocuses his/her attention too much on the driving operation. Incontrast, when the driver has little driving experience, that is, whenthe driver has low driving proficiency, the driver's visual field isnarrow and thus the visual field range is narrow. The reason why thevisual field range is narrow is that a driver having low proficiencytends to be strained and focus his/her attention too much on the drivingoperation and thus the ratio of attention directed towards the vision islow. Further, the visual recognition level is high in a region close tothe line of sight and decreases as the position moves from the line ofsight towards the edge of the visual field range. For theabove-described reasons, the visual recognition level calculation unit104 is capable of estimating the visual recognition level of the driverin the visual field range centering at the line of sight represented bythe sight line direction information D2 on the driver and in eachsection in the visual field range based on the driving skill of thedriver.

Further, the driver's information processing capability in regard to thesight line direction is evaluated by using information quantityintegrated in a time axis space. Therefore, when the visual recognitionlevel calculation unit 104 recognizes that the driver is continuouslyviewing in the same direction other than the direction of a centerposition of the visual field range, it can be considered that thedriver's focusing of attention on the center position of the visualfield range has decreased and the visual recognition level at the centerposition of the visual field range has decreased. Therefore, in the casewhere it is recognized that the driver is continuously viewing in thesame direction other than that of the center position of the visualfield range, the visual recognition level calculation unit 104 cancorrect the visual recognition level in each of the plurality ofsections based on the direction in which the driver is continuouslyviewing for a long time and the sight line direction.

Furthermore, it has been confirmed that the visual field range and thevisual recognition level of the driver change depending on the driver'shealth condition or actions other than the driving actions. Therefore,the visual recognition level calculation unit 104 may correct the visualfield range and the visual recognition level obtained based on thedriving skill depending on the driver's condition.

Moreover, when the traveling environment indicated by the travelingenvironment information D3 is an environment with high density ofvehicles such as congestion, for example, the visual recognition levelis considered to decrease in the whole of the visual field range sincethere are a lot of events that should be considered in regard to thedriving actions, such as adjoining vehicles and a pedestrian rushing outonto the road. Therefore, the visual recognition level calculation unit104 may correct the estimated visual field range and visual recognitionlevel based on the traveling environment information D3.

In addition, in an environment in which obstruction or focusing ofattention in regard to a particular direction is seen, such as anenvironment in which sunlight comes in from a certain direction or thevehicle goes around a curve, the visual recognition level calculationunit 104 may correct the estimated visual field range and visualrecognition level based on the direction of sunlight in the visual fieldrange, the traveling direction of the vehicle, and so forth.

As shown in FIG. 5, the visual recognition level indicated by the visualrecognition level map 40 is represented as a numerical value (density inFIG. 5) in each of the plurality of rectangular sections arranged in agrid in the visual field range in front as viewed from the driver.First, the visual recognition level calculation unit 104 obtains acenter of the driver's viewpoint position from the sight line directioninformation D2 and calculates an intersection point of athree-dimensional straight line in the direction of a sight line centerand the visual field range in front. The sight line center variesdepending on the method of obtaining the sight line directioninformation D2. For example, when the sight line direction is obtainedbased on the pupils of both eyes, the direction of the sight line centeris the direction of a straight line connecting an intersection point ofsight line vectors of both eyes and a midpoint of three-dimensionalpositions of both eyes. A vector in the direction of the sight linecenter is a sight line center vector.

Subsequently, the visual recognition level calculation unit 104calculates the visual recognition level of the rectangular sectioncorresponding to the position of the intersection point of the visualfield range in front and the straight line in the direction of the sightline center. The calculation of the visual recognition level of eachsection is made by a process of subtracting a decrease component of thevisual recognition level including the traveling environment or thedriver's condition as a variable from a maximum value of the visualrecognition level. For example, when road condition around the vehicleas the traveling environment is congested, the number of objects thatthe driver should view increases, and thus the decrease component of thevisual recognition level takes on a large value. For example, in anenvironment in which the driver's ability to concentrate decreases suchas in a case where the driver is in mental condition with impatience,the driver's physical health condition is poor or the driver isconversing with a passenger, the decrease component of the visualrecognition level takes on a large value. For example, when the driver'sdriving skill is low, the driver is less likely to pay attention to thesurroundings, and thus the decrease component of the visual recognitionlevel takes on a small value and the visual recognition level in thesection in the direction of the sight line center becomes high.

Subsequently, the visual recognition level calculation unit 104successively calculates the visual recognition levels of the sectionsaround the section at the center position of the visual field rangebased on the visual recognition level of the section at the centerposition. The visual recognition level in the visual field range of thehuman gradually decreases like a pattern of concentric circles as theposition separates from the center position. Therefore, the visualrecognition levels of the sections around the section at the centerposition are calculated by regarding the center position as the centerof the concentric circles and based on the value of the visualrecognition level of the section at the center position by using acalculation formula as a function including a variable representing theradius of a concentric circle. Further, this calculation formula changesdepending on the driving environment, the driver's condition and soforth. For example, when the driving environment is congested, thedriver often pays attention to the surroundings, and thus the ratio ofthe decrease in the visual recognition level with the increase in theradius is low. For example, the driver's mental condition or physicalcondition influences the driver's ability to concentrate, and thus theratio of the decrease in the visual recognition level varies dependingon the driver's condition. For example, when the driver's driving skillis low, the driver is incapable of sufficiently paying attention to thesurroundings, and thus the ratio of the decrease in the visualrecognition level becomes high in peripheral sections.

Further, the visual recognition level calculation unit 104 may usedifferent values of the visual recognition level for the inside and theoutside of a concentric circle at a predetermined radius from the sightline center. In the visual properties of the human, the central visionregion is generally regarded as a region within approximately 30 degreesto the left and right with reference to the sight line center, and avisual field range at larger angles is referred to as the peripheralvision region. It is generally said that a great difference occursbetween the central vision region and the peripheral vision region inthe visual recognition level in cases like discrimination betweenconditions by a human. Therefore, the visual recognition levelcalculation unit 104 may separately handle sections situated in thecentral vision region from the sight line direction in thethree-dimensional space and sections situated in the peripheral visionregion and perform the calculation of the visual recognition levels in away adapted to the visual properties of the driver by changing acalculation formula to be subtracted (i.e., calculation formularepresenting the decrease component of the visual recognition level).

Furthermore, the visual recognition level calculation unit 104 mayexecute a visual recognition level correction process based on theenvironment outside the vehicle. For example, when intense light from aparticular direction, such as backlight, light from a street lamp atnight or light from an illuminator in a tunnel, enters the visual field,the visual recognition level in regions around the intense light dropssignificantly. The visual recognition level calculation unit 104 cancarry out calculation of the visual recognition level that is robust tothe traveling environment by making a correction, by subtracting adecrease component of the visual recognition level from the visualrecognition level obtained from the calculation formula, based on thetraveling environment information D3 including a result of detecting anobject outside the vehicle or detecting environment around the vehicle.

(1-7) Display Control Unit 105

The display control unit 105 generates image information D5, forperforming display control suitable for the driver, based on the displayinformation D0 and the visual recognition level D4. The displayinformation D0 is visual presentation information regarding the drivingactions. The display information D0 includes, for example, a speedindication on the panel display device 12, information indicatingvehicle condition, navigation information, alert information at times ofdriving, and so forth. Further, the display information D0 may includeinformation indicating an importance level of each item of presentationinformation used in the display control.

The display control unit 105 provides display signals to the HUD device11, the panel display device 12, the side mirror display devices 13 and14 and the rearview mirror display device 15 as the various types ofdisplay devices. The display control unit 105 performs display controlsuitable for the driver based on the display information D0 and thevisual recognition level D4. The display control is implemented byexecuting a control process of controlling the display content and thedisplay method based on the visual recognition levels D4 at the displaypositions of the HUD device 11, the panel display device 12, the sidemirror display devices 13 and 14 and the rearview mirror display device15. For example, the display control unit 105 arranges images of aplurality of items of display information D0 at positions at high visualrecognition levels according to the importance levels of the pluralityof items of display information D0. Further, the display control unit105 may change the display position, the display color, the display timeand the display content of each of the plurality of items of displayinformation D0 depending on the importance level, for example.

The display control unit 105 may arrange the images so as to make itpossible to intuitively notify the driver of information regarding adirection included in the display information D0, a relevance betweenthe display information D0 and another item of information, or the like.For example, when an object to which the driver should pay attentionexists in a certain direction, the display control unit 105 may arrangean image based on the display information D0 in the same direction asthat of the object. Further, the display control unit 105 may carry outthe display so that the display information D0 and another item ofinformation related to the display information D0 are connected to eachother in an image. In this case, the display control unit 105 is capableof making the driver appropriately perceive an image displayed at aposition at a low visual recognition level.

Further, if an image based on the display information D0 is arranged ata position at a high visual recognition level, there is a possibilitythat visual attention in the driving action is obstructed. Therefore,the display control unit 105 may set an upper limit to the area of adisplay region that can be displayed in a region whose visualrecognition level D4 is higher than or equal to a predeterminedthreshold value. In this case, the visual attention in the drivingaction becomes less likely to be obstructed by an image displayed at aposition at a high visual recognition level.

Furthermore, the display control unit 105 can prompt the driver tointuitively recognize the displayed information by changing the color ofthe displayed image depending on the importance level of the displayinformation D0. When an image is displayed by the HUD device 11, thedisplay control unit 105 can make the image be more perceptible by usinga color greatly different from the color of the scene outside thevehicle depending on the environment outside the vehicle. Further, sincethe color of the outside world viewed by the driver changes depending oninsolation environment and road environment outside the vehicle, thedisplay control unit 105 can display an image easily perceptible to thedriver by displaying the image in a color greatly different from thecolor of the outside world viewed by the driver.

When displaying an image based on display information D0 at a highimportance level for a long time, the display control unit 105 candisplay the image at a position at a low visual recognition level eventhough the display information D0 is at the high importance level. Whendisplaying an image based on display information D0 requiring urgency,the display control unit 105 can enhance the alerting effect bydisplaying the image at a position at a high visual recognition leveljust for a time period enabling the recognition of the image. Thedisplay control unit 105 can obtain the shortest display time capable ofachieving a sufficiently strong alerting effect by calculating how theamount of accumulation of visually perceived information variesdepending on the display time and the sight line direction.

Further, by using a figure such as an arrow, an illustration or an iconas the display content, the display control unit 105 can shorten thetime the driver takes to recognize the displayed information compared tocases of displaying text. Furthermore, the display control unit 105determines a necessity level of the display information D0 indicating towhat extent the information is necessary for the driver, according tothe contents of the display information D0, arranges an image of afigure that the driver can understand at a glance, such as anillustration or an icon, at a position at a low visual recognition levelif the necessity level is low, and arranges an image of a sentence orthe like at a position at a high visual recognition level if thenecessity level is high.

FIG. 6 is a diagram showing an example of an image displayed based oncontrol executed by the display control device 100 according to thefirst embodiment. FIG. 6 shows an example of the contents of displaycontrol assuming a driver whose driving skill is low. In general, thevisual field range is narrow in cases of a driver whose driving skill islow. FIG. 6 shows a case where the sight line center is at the center ofa driving direction as the vehicle's traveling direction and a part ofthe HUD device 11 or the panel display device 12 can be perceived as theperipheral vision. Thus, when the display information D0 is at a highimportance level like information regarding an object to which thedriver needs to pay attention, the display control unit 105 increasesthe visual recognition level by arranging the image at a position on theHUD device 11 and close to the sight line direction.

Further, the display control unit 105 produces a display that isintuitively recognizable by increasing the ratio of figure display of anicon or the like relative to text display. Furthermore, for informationat a low importance level in the display information D0, the displaycontrol unit 105 executes display control that does not hinder thedriving action, such as position control of arranging the image in theperipheral vision region or control of setting a display timeproportional to the importance level and changing the display contentdepending on the time.

FIG. 7 is a diagram showing an example of an image displayed based oncontrol executed by the display control device 100 according to thefirst embodiment. FIG. 7 shows the contents of display control on theassumption that the driver's driving skill is high. In general, thevisual field range is wide in cases of a driver whose driving skill ishigh. FIG. 7 shows a case where the sight line center is at the centerof the driving direction, upper halves of the HUD device 11 and thepanel display device 12 can be perceived as the central vision, andparts of the side mirror display devices 13 and 14 can be perceived asthe peripheral vision. Since the central visual field is wide based onthe visual recognition levels, the display control unit 105 carries outa display that is unlikely to cause sight line guidance by arranging anavigation icon 41 and destination display information 44 in an edgepart of a display surface of the HUD device 11, that is, in a region ata low visual recognition level. Further, the display control unit 105arranges congestion level display information 45 in an upper part of thepanel display device 12, or arranges an alert display icon 43 in an edgepart of the side mirror display device 14, or the like, which enablesthe driver to perceive visual information at a low importance level inthe wide peripheral visual field without undergoing the sight lineguidance.

(1-8) Effect

As described above, with the display control device 100, the displaycontrol method or the display control program according to the firstembodiment, the display position and the display method of the imagepresented to the driver can be determined according to the driver'sdriving skill, the driver's sight line direction and the environmentaround the vehicle, and thus the display control can be executed so thatthe image is displayed at a position and by a method appropriate foreach driver. For example, a driver whose driving skill is high canrecognize the image information precisely and without having the visualfield obstructed by the displayed image. A driver whose driving skill islow can focus his/her attention on the driving without overlooking thedisplayed image and without being forced to widely move the line ofsight.

(2) Second Embodiment

A display control device according a second embodiment has aconfiguration similar to that of the display control device 100according to the first embodiment shown in FIG. 3. Further, a displaycontrol method executed by the display control device according thesecond embodiment is a process similar to the display control methodaccording to the first embodiment shown in FIG. 4. Therefore, FIG. 1 toFIG. 4 are referred to in the description of the second embodiment. Inthe second embodiment, a description will be given of a display controlmethod at the time of the driving action by use of the side mirrordisplay devices 13 and 14 and the rearview mirror display device 15 thatdisplay images of the outside of the vehicle.

FIG. 8 is a diagram showing an example of an image displayed based oncontrol executed by the display control device according to the secondembodiment. FIG. 8 shows an example of display control of the sidemirror display device 13 and the HUD device 11. FIG. 9 is a diagramshowing an example of an image displayed based on control executed bythe display control device according to the second embodiment. FIG. 9shows an example of display control of the rearview mirror displaydevice 15 and the HUD device 11. In general, the visual field range atthe time of a right turn driving action changes depending on the drivingsituation from a front right direction to the direction of the sidewindow. At the time of a reverse driving action, regions around the leftand right side mirror display devices 13 and 14 and the rearview mirrordisplay device 15 can be included in the visual field range. Further,these visual field ranges vary depending on the driver's driving skill,the driver's habit, or both of these factors.

In the second embodiment, the visual recognition level calculation unit104 calculates the visual recognition level D4 in the right turn drivingaction by preestimating movement of the visual field range with time atthe time of the right turn driving action based on the skill informationD1 as the result of the estimation by the skill estimation unit 101 orthe sight line direction information D2 as the result of the detectionby the sight line detection unit 102. Here, for the preestimation of themovement of the visual field range with time, the visual recognitionlevel calculation unit 104 accumulates information indicating movementof the sight line direction and the driving action in similar drivingactions, and calculates the visual field range and the visualrecognition level D4 by averaging the accumulated information. Further,since the sight line direction and the driving action change dependingon the environment outside the vehicle, the visual recognition levelcalculation unit 104 is desired to average a certain number of pieces ofinformation greater than or equal to a predetermined number of times.Furthermore, in order to exclude influence of information transitioncaused by slight movements, the visual recognition level calculationunit 104 may perform a filtering process on the information indicatingthe sight line direction and the information indicating the drivingaction changing depending on the environment outside the vehicle.

By the above-described method, even when the driver's viewpoint positionchanges in the right turn driving action, the display control unit 105can arrange information at a high importance level in the visible visualfield and arrange information at a low importance level so that theinformation is visible only at a certain viewpoint position in the rightturn driving action.

FIG. 8 shows an example of the display control in the right turn drivingaction. The navigation icon 41 can be displayed so that the icon can beperceived in the central vision region only at the viewpoint position atthe start of the right turn driving action and thereafter the icon neverobstructs the visual field in the right turn driving action. Further, byperforming arrangement control so that the alert display icons 43 and 46at high information importance levels are in the visual field range atany time and position in the right turn driving action, the time whenthe driver recognizes an object that needs to be paid attention to canbe made earlier.

FIG. 9 shows an example of display control when the driver is viewingthe rearview mirror display device 15 at the time of the reverse drivingaction. The alert display icons 43 and 46 are display information formaking the driver recognize an object that needs to be paid attention towhen the object has appeared in a right rear direction. In this example,when the driver is viewing the rearview mirror display device 15, anicon not obstructing other display information is arranged on therearview mirror display device 15. In this case, by performing the icondisplay so that the driver can view the icon in a display region of theHUD device 11 situated in the peripheral visual field when the driverhas changed the viewpoint position to the rightward direction, thedriver is relieved of the need to move his/her body or line of sight forthe purpose of viewing the icon at the time of the reverse drivingaction.

(3) Third Embodiment

A display control device according a third embodiment has aconfiguration similar to that of the display control device 100according to the first embodiment shown in FIG. 3. Further, a displaycontrol method executed by the display control device according thethird embodiment is a process similar to the display control method ofthe display control device 100 according to the first embodiment shownin FIG. 4. Therefore, FIG. 1 to FIG. 4 are referred to in thedescription of the third embodiment. In the third embodiment, adescription will be given of a display control method used when sightline movement necessary for a driving action has not been performedappropriately.

FIG. 10 is a diagram showing an example of an image displayed based oncontrol executed by the display control device according to the thirdembodiment. FIG. 10 shows an example of a display for guiding thedriver's line of sight to a direction of a position where an object 47that needs to be paid attention to exists outside the range of thevisual recognition level map 40 when the visual recognition levels D4 ofthe driver have been determined as in the visual recognition level map40 in FIG. 5.

In the third embodiment, when the viewpoint position needs to be pointedoutside the range of the visual recognition level map 40, the displaycontrol unit 105 displays the alert display icon 46 at a position in thevisual field range and close to the direction of the guidance. With thismethod, the sight line guidance display can be carried out withoutobstructing the driving action that the driver has in mind and theperception by the central visual field.

(4) Modification

Further, the configurations of the display control devices in the firstto third embodiments described above can be appropriately combined witheach other.

DESCRIPTION OF REFERENCE CHARACTERS

1: driving support device, 10: front windshield, 11: HUD device, 12:panel display device, 13: side mirror display device, 14: side mirrordisplay device, 15: rearview mirror display device, 21: in-vehiclecamera, 22: vehicle-outside camera, 31: processor, 32: memory, 40:visual recognition level map, 41: navigation icon, 42: alert displayicon, 43: alert display icon, 44: destination display information, 45:congestion level display information, 46: alert display icon, 100:display control device, 101: skill estimation unit, 102: sight linedetection unit, 103: environment information acquisition unit, 104:visual recognition level calculation unit, 105: display control unit,D0: display information, D1: driving skill information, D2: sight linedirection information, D3: traveling environment information, D4: visualrecognition level, D5: image information.

1. A display control device comprising: processing circuitry to acquire traveling environment information indicating environment around a vehicle driven by a driver; to estimate driving skill of the driver and to generate driving skill information indicating the driving skill; to detect a direction of a line of sight of the driver and to generate sight line direction information indicating the direction of the line of sight; to accumulate information on movements of the sight line direction occurring at times of driving actions of the driver and driving action information on the driver in a memory, the information on the movements being obtained by the sight line direction information; to calculate a visual field range indicating a region that can be visually recognized by the driver and a visual recognition level indicating ease of recognition of each of a plurality of sections in the visual field range by the driver's vision based on the information on the movements of the sight line direction and the driving action information accumulated in the memory and at least one of the traveling environment information and the driving skill information; and to receive display information to be presented to the driver and to control at least one of a display position and a display method of an image on a basis of the display information based on the display information, the visual field range and the visual recognition level.
 2. The display control device according to claim 1, wherein the processing circuitry calculates the visual recognition level based on the sight line direction information on the driver, the driving skill information and the traveling environment information.
 3. (canceled)
 4. The display control device according to claim 1, wherein the processing circuitry edits the position and the display method of displaying the image on the basis of the display information depending on the acquired visual recognition level and an importance level of the display information.
 5. The display control device according to claim 1, wherein the processing circuitry executes control for producing a display that guides the driver's line of sight from the driver's visual field range to a sight line direction in which the driver should view according to the driver's visual field range and the acquired visual recognition level and information indicating the sight line direction in which the driver should view included in the display information.
 6. The display control device according to claim 1, wherein the processing circuitry acquires position information regarding the vehicle, weather around the vehicle and road environment information.
 7. The display control device according to claim 1, wherein the processing circuitry detects an object existing around the vehicle based on information acquired from a camera that captures an image of a scene outside the vehicle or an infrared ray sensor.
 8. The display control device according to claim 1, wherein the processing circuitry acquires and accumulates driving actions of the driver and estimates the driving skill from a frequency of a predetermined driving action.
 9. The display control device according to claim 8, wherein the processing circuitry acquires biological information on the driver and corrects the driving skill information based on the biological information.
 10. The display control device according to claim 8, wherein the processing circuitry acquires the driver's driving experience, driving history records, health condition or eyesight information and corrects the driving skill information based on one or more items of information among the driving experience, the driving history records, the health condition and the eyesight information.
 11. A display control method comprising: acquiring traveling environment information indicating environment around a vehicle driven by a driver; estimating driving skill of the driver and generating driving skill information indicating the driving skill; detecting a direction of a line of sight of the driver and generating sight line direction information indicating the direction of the line of sight; accumulating information on movements of the sight line direction occurring at times of driving actions of the driver and driving action information on the driver in a memory, the information on the movements being obtained by the sight line direction information; calculating a visual field range indicating a region that can be visually recognized by the driver and a visual recognition level indicating ease of recognition of each of a plurality of sections in the visual field range by the driver's vision based on the accumulated information on the movements of the sight line direction and the accumulated driving action information and at least one of the traveling environment information and the driving skill information; and receiving display information to be presented to the driver and controlling at least one of a display position and a display method of an image on a basis of the display information based on the display information, the visual field range and the visual recognition level.
 12. A non-transitory computer-readable storage medium storing a display control program that causes a computer to execute: acquiring traveling environment information indicating environment around a vehicle driven by a driver; estimating driving skill of the driver and generating driving skill information indicating the driving skill; detecting a direction of a line of sight of the driver and generating sight line direction information indicating the direction of the line of sight; accumulating information on movements of the sight line direction occurring at times of driving actions of the driver and driving action information on the driver in a memory, the information on the movements being obtained by the sight line direction information; calculating a visual field range indicating a region that can be visually recognized by the driver and a visual recognition level indicating ease of recognition of each of a plurality of sections in the visual field range by the driver's vision based on the accumulated information on the movements of the sight line direction and the accumulated driving action information and at least one of the traveling environment information and the driving skill information; and receiving display information to be presented to the driver and controlling at least one of a display position and a display method of an image on a basis of the display information based on the display information, the visual field range and the visual recognition level. 